{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "title: Testing\n",
    "sidebar_position: 2\n",
    "description: Testing Mojo programs.\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mojo includes a framework for developing and executing unit tests. The framework\n",
    "also supports testing code examples in the\n",
    "[documentation strings](/mojo/manual/basics#code-comments)\n",
    "(also known as *docstrings*) of your API references. The Mojo testing framework\n",
    "consists of a set of assertions defined as part of the\n",
    "[Mojo standard library](/mojo/lib) and the\n",
    "[`mojo test`](/mojo/cli/test) command line tool.\n",
    "\n",
    "## Get started\n",
    "\n",
    "Let’s start with a simple example of writing and running Mojo tests.\n",
    "\n",
    "### 1. Write tests\n",
    "\n",
    "For your first example of using the Mojo testing framework, create a file named\n",
    "`test_quickstart.mojo` containing the following code:\n",
    "\n",
    "```mojo\n",
    "# Content of test_quickstart.mojo\n",
    "from testing import assert_equal\n",
    "\n",
    "def inc(n: Int) -> Int:\n",
    "    return n + 1\n",
    "\n",
    "def test_inc_zero():\n",
    "    # This test contains an intentional logical error to show an example of\n",
    "    # what a test failure looks like at runtime.\n",
    "    assert_equal(inc(0), 0)\n",
    "\n",
    "def test_inc_one():\n",
    "    assert_equal(inc(1), 2)\n",
    "```\n",
    "\n",
    "In this file, the `inc()` function is the test *target*. The functions whose\n",
    "names begin with `test_` are the tests. Usually the target should be in a\n",
    "separate source file from its tests, but you can define them in the same file\n",
    "for this simple example.\n",
    "\n",
    "A test function *fails* if it raises an error when executed, otherwise it\n",
    "*passes*. The two tests in this example use the `assert_equal()` function,\n",
    "which raises an error if the two values provided are not equal.\n",
    "\n",
    ":::note\n",
    "\n",
    "The implementation of `test_inc_zero()` contains an intentional logical error\n",
    "so that you can see an example of a failed test when you execute it in the\n",
    "next step of this tutorial. \n",
    "\n",
    ":::\n",
    "\n",
    "### 2. Execute tests\n",
    "\n",
    "Then in the directory containing the file, execute the following command in your\n",
    "shell:\n",
    "\n",
    "```bash\n",
    "mojo test test_quickstart.mojo\n",
    "```\n",
    "\n",
    "You should see output similar to this (note that this example elides the full\n",
    "filesystem paths from the output shown):\n",
    "\n",
    "```\n",
    "Testing Time: 1.193s\n",
    "\n",
    "Total Discovered Tests: 2\n",
    "\n",
    "Passed : 1 (50.00%)\n",
    "Failed : 1 (50.00%)\n",
    "Skipped: 0 (0.00%)\n",
    "\n",
    "******************** Failure: 'ROOT_DIR/test_quickstart.mojo::test_inc_zero()' ********************\n",
    "\n",
    "Unhandled exception caught during execution\n",
    "\n",
    "Error: At ROOT_DIR/test_quickstart.mojo:8:17: AssertionError: `left == right` comparison failed:\n",
    "   left: 1\n",
    "  right: 0\n",
    "\n",
    "********************\n",
    "```\n",
    "\n",
    "The output starts with a summary of the number of tests discovered, passed,\n",
    "failed, and skipped. Following that, each failed test is reported along with\n",
    "its error message.\n",
    "\n",
    "### Next steps\n",
    "\n",
    "- [The `testing` module](#the-testing-module) describes the assertion\n",
    "functions available to help implement tests.\n",
    "- [Writing unit tests](#writing-unit-tests) shows how to write unit tests and\n",
    "organize them into test files.\n",
    "- [The `mojo test` command](#the-mojo-test-command) describes how to execute\n",
    "and collect lists of tests.\n",
    "- [Writing API documentation tests](#writing-api-documentation-tests)\n",
    "discusses how to use the Mojo testing framework to test code examples in your\n",
    "API documentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## The `testing` module\n",
    "\n",
    "The Mojo standard library includes a [`testing`](/mojo/stdlib/testing/testing/)\n",
    "module that defines several assertion functions for implementing tests. Each\n",
    "assertion returns `None` if its condition is met or raises an error if it isn’t.\n",
    "\n",
    "- [`assert_true()`](/mojo/stdlib/testing/testing/assert_true):\n",
    "Asserts that the input value is `True`.\n",
    "- [`assert_false()`](/mojo/stdlib/testing/testing/assert_false):\n",
    "Asserts that the input value is `False`.\n",
    "- [`assert_equal()`](/mojo/stdlib/testing/testing/assert_equal):\n",
    "Asserts that the input values are equal.\n",
    "- [`assert_not_equal()`](/mojo/stdlib/testing/testing/assert_not_equal):\n",
    "Asserts that the input values are not equal.\n",
    "- [`assert_almost_equal()`](/mojo/stdlib/testing/testing/assert_almost_equal):\n",
    "Asserts that the input values are equal up to a tolerance.\n",
    "\n",
    "The boolean assertions report a basic error message when they fail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unhandled exception caught during execution"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: At Expression [1] wrapper:14:16: AssertionError: condition was unexpectedly False\n"
     ]
    }
   ],
   "source": [
    "from testing import *\n",
    "assert_true(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each function also accepts an optional `msg` keyword argument for providing a\n",
    "custom message to include if the assertion fails."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unhandled exception caught during execution"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: At Expression [2] wrapper:14:16: AssertionError: paradoxes are not allowed\n"
     ]
    }
   ],
   "source": [
    "assert_true(False, msg=\"paradoxes are not allowed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For comparing floating point values you should use `assert_almost_equal()`,\n",
    "which allows you to specify either an absolute or relative tolerance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unhandled exception caught during execution"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: At Expression [3] wrapper:15:24: AssertionError: 3.3333333333333335 is not close to 3.3300000000000001 with a diff of 0.0033333333333334103 (close but no cigar)\n"
     ]
    }
   ],
   "source": [
    "result = 10 / 3\n",
    "assert_almost_equal(result, 3.33, atol=0.001, msg=\"close but no cigar\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The testing module also defines a context manager,\n",
    "[`assert_raises()`](/mojo/stdlib/testing/testing/assert_raises),\n",
    "to assert that a given code block correctly raises an expected error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unhandled exception caught during execution"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test passes because the error is raised\n",
      "Test fails because the error isn't raised\n",
      "Error: AssertionError: Didn't raise at Expression [4] wrapper:18:23\n"
     ]
    }
   ],
   "source": [
    "def inc(n: Int) -> Int:\n",
    "    if n == Int.MAX:\n",
    "         raise Error(\"inc overflow\")\n",
    "    return n + 1\n",
    "\n",
    "print(\"Test passes because the error is raised\")\n",
    "with assert_raises():\n",
    "    _ = inc(Int.MAX)\n",
    "\n",
    "print(\"Test fails because the error isn't raised\")\n",
    "with assert_raises():\n",
    "    _ = inc(Int.MIN)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::note\n",
    "\n",
    "The example above assigns the return value from `inc()` to a\n",
    "[*discard pattern*](/mojo/manual/lifecycle/death#explicit-lifetimes).\n",
    "Without it, the Mojo compiler detects that the return value is unused and\n",
    "optimizes the code to eliminate the function call.\n",
    "\n",
    ":::\n",
    "\n",
    "You can also provide an optional `contains` argument to `assert_raises()` to\n",
    "indicate that the test passes only if the error message contains the substring\n",
    "specified. Other errors are propagated, failing the test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unhandled exception caught during execution"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test passes because the error contains the substring\n",
      "Test fails because the error doesnt contain the substring\n",
      "Error: invalid value\n"
     ]
    }
   ],
   "source": [
    "print(\"Test passes because the error contains the substring\")\n",
    "with assert_raises(contains=\"required\"):\n",
    "    raise Error(\"missing required argument\")\n",
    "\n",
    "print(\"Test fails because the error doesnt contain the substring\")\n",
    "with assert_raises(contains=\"required\"):\n",
    "    raise Error(\"invalid value\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing unit tests\n",
    "\n",
    "A Mojo unit test is simply a function that fulfills all of these requirements:\n",
    "\n",
    "- Has a name that starts with `test_`.\n",
    "- Accepts no arguments.\n",
    "- Returns either `None` or a value of type `object`.\n",
    "- Raises an error to indicate test failure.\n",
    "- Is defined at the module scope, not as a Mojo struct method.\n",
    "\n",
    "You can use either `def` or `fn` to define a test function. Because a test\n",
    "function always raises an error to indicate failure, any test function defined\n",
    "using `fn` must include the `raises` declaration.\n",
    "\n",
    "Generally, you should use the assertion utilities from the Mojo standard library\n",
    "[`testing`](/mojo/stdlib/testing/testing/) module to implement your tests.\n",
    "You can include multiple related assertions in the same test function. However,\n",
    "if an assertion raises an error during execution then the test function returns\n",
    "immediately, skipping any subsequent assertions.\n",
    "\n",
    "You must define your Mojo unit tests in Mojo source files named with a `test`\n",
    "prefix or suffix. You can organize your test files within a directory hierarchy,\n",
    "but the test files must not be part of a Mojo package (that is, the test\n",
    "directories should not contain `__init__.mojo` files).\n",
    "\n",
    "Here is an example of a test file containing three tests for functions defined\n",
    "in a source module named `my_target_module` (which is not shown here).\n",
    "\n",
    "```mojo\n",
    "# File: test_my_target_module.mojo\n",
    "\n",
    "from my_target_module import convert_input, validate_input\n",
    "from testing import assert_equal, assert_false, assert_raises, assert_true\n",
    "\n",
    "def test_validate_input():\n",
    "\tassert_true(validate_input(\"good\"), msg=\"'good' should be valid input\")\n",
    "\tassert_false(validate_input(\"bad\"), msg=\"'bad' should be invalid input\")\n",
    "\n",
    "def test_convert_input():\n",
    "\tassert_equal(convert_input(\"input1\"), \"output1\")\n",
    "\tassert_equal(convert_input(\"input2\"), \"output2\")\n",
    "\n",
    "def test_convert_input_error():\n",
    "\twith assert_raises():\n",
    "\t\t_ = convert_input(\"garbage\")\n",
    "```\n",
    "\n",
    "The unique identity of a unit test consists of the path of the test file and the\n",
    "name of the test function, separated by `::`. So the test IDs from the example\n",
    "above are:\n",
    "\n",
    "- `test_my_target_module.mojo::test_validate_input()`\n",
    "- `test_my_target_module.mojo::test_convert_input()`\n",
    "- `test_my_target_module.mojo::test_convert_error()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `mojo test` command\n",
    "\n",
    "The `mojo` command line interface includes the [`mojo test`](/mojo/cli/test)\n",
    "command for running tests or collecting a list of tests.\n",
    "\n",
    "### Running tests\n",
    "\n",
    "By default, the `mojo test` command runs the tests that you specify using one of\n",
    "the following:\n",
    "\n",
    "- A single test ID with either an absolute or relative file path, to run only\n",
    "that test.\n",
    "- A single absolute or relative file path, to run all tests in that file.\n",
    "- A single absolute or relative directory path, to recurse through that\n",
    "directory hierarchy and run all tests found.\n",
    "\n",
    "If needed, you can optionally use the `-I` option one or more times to append\n",
    "additional paths to the list of directories searched to import Mojo modules and\n",
    "packages. For example, consider a project with the following directory\n",
    "structure:\n",
    "\n",
    "```\n",
    ".\n",
    "├── src\n",
    "│   ├── example.mojo\n",
    "│   └── my_math\n",
    "│       ├── __init__.mojo\n",
    "│       └── utils.mojo\n",
    "└── test\n",
    "    └── my_math\n",
    "        ├── test_dec.mojo\n",
    "        └── test_inc.mojo\n",
    "```\n",
    "\n",
    "From the project root directory, you could execute all of the tests in the\n",
    "`test` directory like this:\n",
    "\n",
    "```\n",
    "$ mojo test -I src test\n",
    "Testing Time: 3.433s\n",
    "\n",
    "Total Discovered Tests: 4\n",
    "\n",
    "Passed : 4 (100.00%)\n",
    "Failed : 0 (0.00%)\n",
    "Skipped: 0 (0.00%)\n",
    "```\n",
    "\n",
    "You could run the tests contained in only the `test_dec.mojo` file like this:\n",
    "\n",
    "```\n",
    "$ mojo test -I src test/my_math/test_dec.mojo\n",
    "Testing Time: 1.175s\n",
    "\n",
    "Total Discovered Tests: 2\n",
    "\n",
    "Passed : 2 (100.00%)\n",
    "Failed : 0 (0.00%)\n",
    "Skipped: 0 (0.00%)\n",
    "```\n",
    "\n",
    "And you could run a single test from a file by providing its fully qualified\n",
    "ID like this:\n",
    "\n",
    "```\n",
    "$ mojo test -I src 'test/my_math/test_dec.mojo::test_dec_valid()'\n",
    "Testing Time: 0.66s\n",
    "\n",
    "Total Discovered Tests: 1\n",
    "\n",
    "Passed : 1 (100.00%)\n",
    "Failed : 0 (0.00%)\n",
    "Skipped: 0 (0.00%)\n",
    "```\n",
    "\n",
    "### Collecting a list of tests\n",
    "\n",
    "By including the `--collect-only` or `--co` option, you can use `mojo test` to\n",
    "discover and print a list of tests. \n",
    "\n",
    "As an example, consider the project structure shown in the\n",
    "[Running tests](#running-tests) section. The following command produces a list\n",
    "of all of the tests defined in the `test` directory hierarchy.\n",
    "\n",
    "```bash\n",
    "mojo test --co test\n",
    "```\n",
    "\n",
    "The output shows the hierarchy of directories, test files, and individual tests\n",
    "(note that this example elides the full filesystem paths from the output shown):\n",
    "\n",
    "```\n",
    "<ROOT_DIR/test/my_math>\n",
    "  <ROOT_DIR/test/my_math/test_dec.mojo>\n",
    "    <ROOT_DIR/test/my_math/test_dec.mojo::test_dec_valid()>\n",
    "    <ROOT_DIR/test/my_math/test_dec.mojo::test_dec_min()>\n",
    "  <ROOT_DIR/test/my_math/test_inc.mojo>\n",
    "    <ROOT_DIR/test/my_math/test_inc.mojo::test_inc_valid()>\n",
    "    <ROOT_DIR/test/my_math/test_inc.mojo::test_inc_max()>\n",
    "```\n",
    "\n",
    "### Producing JSON formatted output\n",
    "\n",
    "By default `mojo test` produces concise, human-readable output. Alternatively\n",
    "you can produce JSON formatted output more suitable for input to other tools by\n",
    "including the `--diagnostic-format json` option.\n",
    "\n",
    "For example, you could run the tests in the `test_quickstart.mojo` file shown\n",
    "in the [Get started](#get-started) section with JSON formatted output using this\n",
    "command:\n",
    "\n",
    "```bash\n",
    "mojo test --diagnostic-format json test_quickstart.mojo\n",
    "```\n",
    "\n",
    "The output shows the detailed results for each individual test and summary\n",
    "results (note that this example elides the full filesystem paths from the\n",
    "output shown):\n",
    "\n",
    "```\n",
    "{\n",
    "  \"children\": [\n",
    "    {\n",
    "      \"duration_ms\": 60,\n",
    "      \"error\": \"Unhandled exception caught during execution\",\n",
    "      \"kind\": \"executionError\",\n",
    "      \"stdErr\": \"\",\n",
    "      \"stdOut\": \"Error: At ROOT_DIR/test_quickstart.mojo:8:17: AssertionError: `left == right` comparison failed:\\r\\n   left: 1\\r\\n  right: 0\\r\\n\",\n",
    "      \"testID\": \"ROOT_DIR/test_quickstart.mojo::test_inc_zero()\"\n",
    "    },\n",
    "    {\n",
    "      \"duration_ms\": 51,\n",
    "      \"error\": \"\",\n",
    "      \"kind\": \"success\",\n",
    "      \"stdErr\": \"\",\n",
    "      \"stdOut\": \"\",\n",
    "      \"testID\": \"ROOT_DIR/test_quickstart.mojo::test_inc_one()\"\n",
    "    }\n",
    "  ],\n",
    "  \"duration_ms\": 1171,\n",
    "  \"error\": \"\",\n",
    "  \"kind\": \"executionError\",\n",
    "  \"stdErr\": \"\",\n",
    "  \"stdOut\": \"\",\n",
    "  \"testID\": \"ROOT_DIR/test_quickstart.mojo\"\n",
    "}\n",
    "```\n",
    "\n",
    "You can also produce JSON output for test collection as well. As an example,\n",
    "consider the project structure shown in the [Running tests](#running-tests)\n",
    "section. The following command collects a list in JSON format of all of the\n",
    "tests defined in the `test` directory hierarchy:\n",
    "\n",
    "```bash\n",
    "mojo test --diagnostic-format json --co test\n",
    "```\n",
    "\n",
    "The output would appear as follows (note that this example elides the full\n",
    "filesystem paths from the output shown):\n",
    "\n",
    "```\n",
    "{\n",
    "  \"children\": [\n",
    "    {\n",
    "      \"children\": [\n",
    "        {\n",
    "          \"id\": \"ROOT_DIR/test/my_math/test_dec.mojo::test_dec_valid()\",\n",
    "          \"location\": {\n",
    "            \"endColumn\": 5,\n",
    "            \"endLine\": 5,\n",
    "            \"startColumn\": 5,\n",
    "            \"startLine\": 5\n",
    "          }\n",
    "        },\n",
    "        {\n",
    "          \"id\": \"ROOT_DIR/test/my_math/test_dec.mojo::test_dec_min()\",\n",
    "          \"location\": {\n",
    "            \"endColumn\": 5,\n",
    "            \"endLine\": 9,\n",
    "            \"startColumn\": 5,\n",
    "            \"startLine\": 9\n",
    "          }\n",
    "        }\n",
    "      ],\n",
    "      \"id\": \"ROOT_DIR/test/my_math/test_dec.mojo\"\n",
    "    },\n",
    "    {\n",
    "      \"children\": [\n",
    "        {\n",
    "          \"id\": \"ROOT_DIR/test/my_math/test_inc.mojo::test_inc_valid()\",\n",
    "          \"location\": {\n",
    "            \"endColumn\": 5,\n",
    "            \"endLine\": 5,\n",
    "            \"startColumn\": 5,\n",
    "            \"startLine\": 5\n",
    "          }\n",
    "        },\n",
    "        {\n",
    "          \"id\": \"ROOT_DIR/test/my_math/test_inc.mojo::test_inc_max()\",\n",
    "          \"location\": {\n",
    "            \"endColumn\": 5,\n",
    "            \"endLine\": 9,\n",
    "            \"startColumn\": 5,\n",
    "            \"startLine\": 9\n",
    "          }\n",
    "        }\n",
    "      ],\n",
    "      \"id\": \"ROOT_DIR/test/my_math/test_inc.mojo\"\n",
    "    }\n",
    "  ],\n",
    "  \"id\": \"ROOT_DIR/test/my_math\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing API documentation tests\n",
    "\n",
    "The Mojo testing framework also supports testing code examples that you include\n",
    "in [docstrings](/mojo/manual/basics#code-comments). This helps to ensure that\n",
    "the code examples in your API documentation are correct and up to date.\n",
    "\n",
    "### Identifying executable code\n",
    "\n",
    "The Mojo testing framework requires you to explicitly identify the code blocks\n",
    "that you want it to execute.\n",
    "\n",
    "In a Mojo docstring, a fenced code block delimited by standard triple-backquotes\n",
    "is a *display-only* code block. It appears in the API documentation, but\n",
    "`mojo test` does not identify it as a test or attempt to execute any of the code\n",
    "in the block.\n",
    "\n",
    "~~~\n",
    "\"\"\" Non-executable code block example.\n",
    "\n",
    "The generated API documentation includes all lines of the following code block,\n",
    "but `mojo test` does not execute any of the code in it.\n",
    "\n",
    "```\n",
    "# mojo test does NOT execute any of this code block\n",
    "a = 1\n",
    "print(a)\n",
    "```\n",
    "\"\"\"\n",
    "~~~\n",
    "\n",
    "In contrast, a fenced code block that starts with the line <code>```mojo</code>\n",
    "not only appears in the API documentation, but `mojo test` treats it as an\n",
    "executable test. The test fails if the code raises any error, otherwise it\n",
    "passes.\n",
    "\n",
    "~~~\n",
    "\"\"\" Executable code block example.\n",
    "\n",
    "The generated API documentation includes all lines of the following code block\n",
    "*and* `mojo test` executes it as a test.\n",
    "\n",
    "```mojo\n",
    "from testing import assert_equals\n",
    "\n",
    "b = 2\n",
    "assert_equals(b, 2)\n",
    "```\n",
    "\"\"\"\n",
    "~~~\n",
    "\n",
    "Sometimes you might want to execute a line of code as part of the test but *not*\n",
    "display that line in the API documentation. To achieve this, prefix the line of\n",
    "code with `%#`. For example, you could use this technique to omit `import`\n",
    "statements and assertion functions from the documentation.\n",
    "\n",
    "~~~\n",
    "\"\"\" Executable code block example with some code lines omitted from output.\n",
    "\n",
    "The generated API documentation includes only the lines of code that do *not*\n",
    "start with `%#`. However, `mojo test` executes *all* lines of code.\n",
    "\n",
    "```mojo\n",
    "%# from testing import assert_equal\n",
    "c = 3\n",
    "print(c)\n",
    "%# assert_equal(c, 3)\n",
    "```\n",
    "\"\"\"\n",
    "~~~\n",
    "\n",
    "### Documentation test suites and scoping\n",
    "\n",
    "The Mojo testing framework treats each docstring as a separate *test suite*.\n",
    "In other words, a single test suite could correspond to the docstring for an\n",
    "individual package, module, function, struct, struct method, etc.\n",
    "\n",
    "Each executable code block within a given docstring is a single test of the same\n",
    "test suite. The `mojo test` command executes the tests of a test suite\n",
    "sequentially in the order that they appear within the docstring. If a test\n",
    "within a particular test suite fails, then all subsequent tests within the same\n",
    "test suite are skipped.\n",
    "\n",
    "All tests within the test suite execute in the same scope, and test execution\n",
    "within that scope is stateful. This means, for example, that a variable created\n",
    "within one test is then accessible to subsequent tests in the same test suite.\n",
    "\n",
    "~~~\n",
    "\"\"\" Stateful example.\n",
    "\n",
    "Assign 1 to the variable `a`:\n",
    "\n",
    "```mojo\n",
    "%# from testing import assert_equal\n",
    "a = 1\n",
    "%# assert_equal(a, 1)\n",
    "```\n",
    "\n",
    "Then increment the value of `a` by 1:\n",
    "\n",
    "```mojo\n",
    "a += 1\n",
    "%# assert_equal(a, 2)\n",
    "```\n",
    "\"\"\"\n",
    "~~~\n",
    "\n",
    ":::note\n",
    "\n",
    "Test suite scopes do *not* nest. In other words, the test suite scope of a\n",
    "module is completely independent of the test suite scope of a function or struct\n",
    "defined within that module. For example, this means that if a module’s test\n",
    "suite creates a variable, that variable is *not* accessible to a function’s test\n",
    "suite within the same module.\n",
    "\n",
    ":::\n",
    "\n",
    "### Documentation test identifiers\n",
    "\n",
    "The format of a documentation test identifier is `<path>@<test-suite>::<test>`.\n",
    "This is best explained by an example. Consider the project structure shown in\n",
    "the [Running tests](#running-tests) section. The source files in the `src`\n",
    "directory might contain docstrings for the `my_math` package, the `utils.mojo`\n",
    "module, and the individual functions within that module. You could collect the\n",
    "full list of tests by executing:\n",
    "\n",
    "```\n",
    "mojo test --co src\n",
    "```\n",
    "\n",
    "The output shows the hierarchy of directories, test files, and individual tests\n",
    "(note that this example elides the full filesystem paths from the output shown):\n",
    "\n",
    "```\n",
    "<ROOT_DIR/src/my_math>\n",
    "  <ROOT_DIR/src/my_math/__init__.mojo>\n",
    "    <ROOT_DIR/src/my_math/__init__.mojo@__doc__>\n",
    "      <ROOT_DIR/src/my_math/__init__.mojo@__doc__::0>\n",
    "      <ROOT_DIR/src/my_math/__init__.mojo@__doc__::1>\n",
    "  <ROOT_DIR/src/my_math/utils.mojo>\n",
    "    <ROOT_DIR/src/my_math/utils.mojo@__doc__>\n",
    "      <ROOT_DIR/src/my_math/utils.mojo@__doc__::0>\n",
    "    <ROOT_DIR/src/my_math/utils.mojo@inc(stdlib\\3A\\3Abuiltin\\3A\\3Aint\\3A\\3AInt).__doc__>\n",
    "      <ROOT_DIR/src/my_math/utils.mojo@inc(stdlib\\3A\\3Abuiltin\\3A\\3Aint\\3A\\3AInt).__doc__::0>\n",
    "      <ROOT_DIR/src/my_math/utils.mojo@inc(stdlib\\3A\\3Abuiltin\\3A\\3Aint\\3A\\3AInt).__doc__::1>\n",
    "    <ROOT_DIR/src/my_math/utils.mojo@dec(stdlib\\3A\\3Abuiltin\\3A\\3Aint\\3A\\3AInt).__doc__>\n",
    "      <ROOT_DIR/src/my_math/utils.mojo@dec(stdlib\\3A\\3Abuiltin\\3A\\3Aint\\3A\\3AInt).__doc__::0>\n",
    "      <ROOT_DIR/src/my_math/utils.mojo@dec(stdlib\\3A\\3Abuiltin\\3A\\3Aint\\3A\\3AInt).__doc__::1>\n",
    "```\n",
    "\n",
    "Several different test suites appear in this result:\n",
    "\n",
    "| Test suite scope | File | Test suite name |\n",
    "|------------------|------|-----------------|\n",
    "| Package          | `src/my_math/__init__.mojo` | `__doc__` |\n",
    "| Module           | `src/my_math/utils.mojo`    | `__doc__` |\n",
    "| Function         | `src/my_math/utils.mojo`    | `inc(stdlib\\3A\\3Abuiltin\\3A\\3Aint\\3A\\3AInt).__doc__` |\n",
    "\n",
    "Then within a specific test suite, tests are numbered sequentially in the order\n",
    "they appear in the docstring, starting with 0."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Mojo",
   "language": "mojo",
   "name": "mojo-jupyter-kernel"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "mojo"
   },
   "file_extension": ".mojo",
   "mimetype": "text/x-mojo",
   "name": "mojo"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
